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Abstract 

In this paper we present the proximal point method for a special class of nonconvex 
function on a Hadamard manifold. The well definedness of the sequence generated by 
the proximal point method is guaranteed. Moreover, it is proved that each accumu- 
lation point of this sequence satisfies the necessary optimality conditions and, under 
additional assumptions, its convergence for a minimizer is obtained. 
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1 Introduction 

The extension of the concepts and techniques of the Mathematical Programming from Eu- 
clidean space to Riemannian manifolds occurs naturally and has been frequently done in 
recent years, with a theoretical purpose as well as to obtain effective algorithms; see P, [2], 
m, no], |I2], US], HE], m], ISg, §0\ and lag. in particular, we observe that, 

these extensions allow the solving of some nonconvex constrained problems in Euclidean 
space. More precisely, nonconvex problems in the classic sense may become convex with 
the introduction of an adequate Riemannian metric on the manifold (see, for example 
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The proximal point algorithm, introduced by Martinet [21] and Rockafellar [27], has been 
extended to different contexts, see [13], [19], [25] and their references. In [13] the proximal 
point method has been generalized in order to solve convex optimization problems of the 
form 



where M is a Hadamard manifold and / : M — )■ M is a convex function (in Riemannian 
sense). The method was described as follows: 



with p° G M an arbitrary point, d the intrinsic Riemannian distance (to be defined later on) 
and {Afc} a sequence of positive numbers. The authors also showed that this extension is 
natural. In [19] the important notion of maximal monotonicity from a multivalued operator 
defined on a Banach space to multivalued vector field defined on a Hadamard manifold has 
been extended. Beside the authors present a general proximal point method to finding singu- 
larity of a multivalued vector field. In particular, as an application of the convergence result 
obtained for the proposed algorithm, constrained optimization problems have been solved. 
With regards to [25] the authors generalized the proximal point method with Bregman dis- 
tance for solving quasiconvex and convex optimization problems on Hadamard manifolds. 
Spingarn has in [31], in particular, developed the proximal point method for the minimization 
of a certain class of nondifferentiable and noncovex functions, namely, lower-C^ functions 
defined on the Euclidean space, see also [14j. Kaplan and Tichatschke in [16j also applied 
the proximal point method for the minimization of a similar class of the ones studied in [l4j 
and [3T|, namely, the maximum of continuously different iable functions. 

Our goal is to study the same class objective functions studied by Kaplan and Tichatschke, 
|16j . in the Riemannian context, applying the proximal point method ([2]) in order to solve 
the problem (JT]) with the objective function in that class. For this purpose, it is necessary to 
study the generalized directional derivative and subdifferential in the Riemannian manifolds 
context. Several works have studied such concepts and presented many useful results in the 
Riemannian nonsmooth optimization context, see for example [3], [18], [22] and [33] . 

The organization of our paper is as follows. In Section 11.11 we define the notations and 
list some results of Riemannian geometry to be used throughout this paper. In Section [2l we 
recall some facts of the convex analysis on Hadamard manifolds. In Section |3] we present some 
properties of the directional derivative of a convex function defined on a Hadamard manifold, 
including a characterization of the directional derivative and of the subdifferential of the 
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maximum of a certain class of convex functions. Here we also present the definition for the 
generalized directional derivative of locally Lipschitz functions (not necessarily convex) and 
an important property of the subdifferential of the maximum of differentiable continuously 
functions. In Section H] we present an application of the proximal point method ([2]) to 
solve the problem ([T]) in the case where the objective function is a real- valued function on 
a Hadamard manifold M (not necessarily convex) given by the maximum of a certain class 
of functions. The main results are the proof of well definition of the sequence generated 
by ([2]), the proof that each accumulation point of this sequence is a stationary point of the 
objective function and, under some additional assumptions, the proof of convergence of that 
sequence to a solution of the problem ([T]). Finally in Section [5] we provide two examples, in 
one of them the curvature of the manifold in consideration is identically zero and the other 
one is nonpositive but not identically zero. In both examples, the proximal point method 
for nonconvex problems, proposal in this paper, is applied. 

1.1 Notation and terminology 

In this section, we introduce some fundamental properties and notations about Riemannian 
geometry. These basics facts can be found in any introductory book on Riemannian geometry, 
such as [6] or [29] . 

Let M be a n-dimentional connected manifold. We denote by TpM the n-dimentional 
tangent space of M at p, by TM = Upi^MTpM tangent bundle of M and by X{M) the space 
of smooth vector fields over M. When M is endowed with a Riemannian metric ( , ), with 
the corresponding norm denoted by || ||, then M is now a Riemannian manifold. Recall that 
the metric can be used to define the length of piecewise smooth curves 7 : [a, 6] — )■ M joining 
p to q, i.e., such that 7(a) = p and 7(6) = q, by: 



and, moreover, by minimizing this length functional over the set of all such curves, we obtain 
a Riemannian distance d{p, q) inducing the original topology on M. The metric induces a 
map / I—)- grad / G X{M) which, for each function smooth over M, associates its gradient via 
the rule (grad/, X) = df{X), X G X{M). Let V be the Levi-Civita connection associated 
with (M, ( , )). In each point p G M, we have a linear map Ax{p) '■ TpM — )■ TpM defined by: 



If X = grad /, where / : M — i- M is a twice differentiable function, then Ax{p) is the Hessian 
of f at p and is denoted by Hess pf. A vector field V along 7 is said to be parallel if 




Ax{p)v = V,X. 



(3) 
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V^iV = 0. If 7' itself is parallel we say that 7 is a geodesic. Given that the geodesic equation 
V yj' = is a second order nonlinear ordinary differential equation, we conclude that the 
geodesic 7 = 7^(.,p) is determined by its position p and velocity v at p. It is easy to check 
that 1 1 7' 1 1 is constant. We say that 7 is normalized if ||7'|| = 1. The restriction of a geodesic 
to a closed bounded interval is called a geodesic segment. A geodesic segment joining p to 
g in M is said to be minimal if its length is equals d{p, q) and the geodesic in question is 
said to be a minimizing geodesic. If 7 is a geodesic joining points p and g in M then, for 
each t G [ci,&], V induces a linear isometry, relative to (, ), P^aj-yit) '■ T^ajM — )■ T^t^M, 
the so-called parallel transport along 7 from 7(a) to •yit). The inverse map of P.^{a)'y(t) is 
denoted by P'f^g^^-yit) '■ ^7(4)^ ~^ T^{a)M. In the particular case of 7 to be the unique geodesic 
segment joining p and g, then the parallel transport along 7 from p to g is denoted by 
Ppg : TpM ^ T,M. 

A Riemannian manifold is complete if the geodesies are defined for any values of t. Hopf- 
Rinow's theorem asserts that if this is the case then any pair of points, say p and g, in M 
can be joined by a (not necessarily unique) minimal geodesic segment. Moreover, (M, d) 
is a complete metric space so that bounded and closed subsets are compact. From the 
completeness of the Riemannian manifold M, the exponential map expp : TpM — )■ M is 
defined by exppV = 7^(1, p), for each p G M. 

We denote by R the curvature tensor defined by R{X, Y) = Vx'^yZ—'Vy'^xZ—'V[y,x]Z, 
with X,Y,Z G X{M), where ^] = YX — XY. Moreover, the sectional curvature with 
respect to X and Y is given by K{X,Y) = {R{X,Y)Y,X)/{\\X\\'^\\X\\^ - {X , Yf), where 
||X|| = (X, X)^. If K{X,Y) ^ for all X and Y, then M is called a Riemannian manifold 
of nonpositive curvature and we use the short notation K ^ 0. 

Theorem 1.1. Let M be a complete, simply connected Riemannian manifold with nonpos- 
itive sectional curvature. Then M is diffeomorphic to the Euclidean space M", n = dimM . 
More precisely, at any point p G M , the exponential map expp is a diffeomorphism. 

Proof. See Lemma 3.2 of [6], p. 149 or Theorem 4.1 of [29], p. 221. □ 

A complete simply connected Riemannian manifold of nonpositive sectional curvature is 
called a Hadamard manifold. Thus Theorem 11.11 states that if M is a Hadamard manifold, 
then M has the same topology and differential structure of the Euclidean space M". Fur- 
thermore, are known some similar geometrical properties of the Euclidean space M", such 
as, given two points there exists an unique geodesic segment that joins them. In this paper, 
all manifolds M are assumed to be Hadamard and finite dimensional. 
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2 Convexity in Hadamard manifolds 



In this section, we introduce some fundamental properties and notations of convex analysis 
on Hadamard manifolds. References of the convex analysis, on the Euclidean space M.^ may 
be found in [15] and on Riemannian manifolds may be found in [7], [13], [26], [29], [30] and 

The set n C M is said to be convex if for any geodesic segment, with end points in Q, 
is contained in Q. Let C M be an open convex set. A function / : M — )■ M is said to 
be convex (respectively, strictly convex) on Q if for any geodesic segment 7 : [a, 6] — )■ f2 the 
composition / o 7 : [a, 6] — ?■ M is convex (respectively, strictly convex). Moreover, a function 
/ : M — )■ M is said to be strongly convex on Q with constant L > if, for any geodesic 
segment 7 : [a, b] — )■ Q, the composition / o 7 : [a,b] — M is strongly convex with constant 
L||7'(0)|p. Take p e M. A vector s G TpM is said to be a suhgradient of / at if: 

/(g) > f{p) + (s,expp^g), 

for any q G M. The set of all subgradients of / at p, df{p), is called the suhdifferential of / 
at p. 

The following result provides a characterization of convexity in the case of differentiable 
functions. 

Proposition 2.1. Let Vt d M he an open convex set and f : M ^ W a differentiable function 
on fl. We say that f is convex on Q if, and only if, for any p E Q: 

f{(l) - f{p) > (grad/(p),expp^g), 

for all q eVL. 

Proof. See Theorem 5.1 of [32j, page 78. □ 

The most important consequence of the previous proposition is that with / being convex, 
any of its critical points are global minimum points. In particular, if M is compact, then / 
is constant. Moreover, G df{p) if, and only if, p is a minimum point of / in M. See, for 
example, [32] . 

Definition 2.1. Let Q G M be an open convex set and X a vector field defined in M. X is 
said to be monotone on VL, if: 

(exp-V, P^pXip) - X{q)) > 0, p, g G (4) 
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where Pqp is the parallel transport along the geodesic joining q to p. If is satisfied with 
strict inequality for all p,q E Q, p ^ q, then X is said to be strictly monotone. Moreover, 
X is strongly monotone if there exists A > such that: 

(exp-ip, P,;iX(p) > Xd\p,q) p, q e Q. (5) 

Remark 2.1. In the particular case that M = MJ^ with the usual metric, inequality (jlj) and 
([5]) becomes, respectively: 

{p-q, X{p) - X{q)) > 0, {p-q, X{p) - X{q)) > X\\p - qf, 

because exp~^ p = p — q and P^^ = I. Therefore the Definition \2.1\ extends the concept of 
monotone operators from R" to Riemannian manifolds. 

Now we present an important example of strong monotone vector field being particularly 
useful in the remainder of this work. 

Take p G M and let exp~^ : M — )■ TpM be the inverse of the exponential map. Note that 
d{q , p) = ||exp~"'^g||, the map . ,p) : M — )■ M is C°° and 

grad^c/^(g,p) = -exp'^p, 

(M is a Hadamard manifold). See, for example, Proposition 4.8 of [29j, p. 108. 

Proposition 2.2. Take p G M. The gradient vector field grad{d'^{ . ,p)/2) is strongly mono- 
tone with A = 1. 

Proof See [?]. □ 

Proposition 2.3. Let Q G M be an open convex set and / : M — )■ M a differentiable function 
on fl. 

(i) f is convex on f2 if and only z/grad/ is monotone on Vt; 

(a) f is strictly convex on Q if and only z/grad/ is strictly monotone on Q; 

(Hi) f is strongly convex on Q if and only i/grad/ is strongly monotone on fl. 

Proof See [7]. □ 

Remark 2.2. Take p G M. From Propositions \2.S\ and \2.3\ it follows that the map d'^{. ,p)/2 
is strongly convex. 
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Proposition 2.4. Let Vt <Z M he a convex set and T C M a compact set. Let : M x T — )■ M 

he a continuous function on Q x T such that tpT- := ip{.,T) : M ^ M. is strongly convex on Q 
with constant L > for all t E T. Then, (p : M -^M. defined hy: 

(f){p) := max^^Ti^{p,r), 

is strongly convex on Q with constant L. In particular, if tpr is convex for all r E T then (p 
is convex on Q. 

Proof. Since T is compact and is continuous, the function (p is well defined. Let 7 : [a, b] — )■ 
n be a geodesic segment. Because tpr is strongly convex with constant L for each t E T, we 
have: 

i^r o ^){at, + (1 - a)t2) < a{^r o l){ti) + (1 - a'K^r o 7)fe) - ^(1 - a)aL\\-f'{0)f, 

for all ti,t2 G [a, b] and a G [0, 1]. Thus, taking the maximum in r in both sides of the above 
inequality, we obtain: 

(0 o 7)(ati + (1 - a)t2) < a{(P o 7)(ti) + (1 _ a)(0 o ^){t2) - ^(1 - a)aL\\-f'{0)\\\ 

which implies that 007: [a,b] — )■ M is strongly convex with constant L||7'(0)|p. So, (p is 
strongly convex on Q with constant L. The proof of the second part is immediate. □ 

Definition 2.2. Let Q G M he an open convex set. A function / : M — )■ M zs said to he 
Lipschitz on Q if there exists a constant L := L{Q) > such that 

\f{p)-fiq)\<Ld{p,q), p,qen. (6) 

Moreover, if for each po E Q there exists L{pq) > and 6 = S{pq) > such that inequality 
^ holds with L = L{po) for all p,q E Bs{po) := {p E Q : d{p,po) < 6}, then f is called 
locally Lipschitz on Q. 

Definition 2.3. Let Q G M he an open convex set and / : M — )■ M a continuously differ- 
entiahle function on Q. The gradient vector field of f , grad/, is said to he Lipschitz with 
constant T > on Q whenever: 

II grad /(g) - Pp, grad/ (p) II <Td{p,q), p,qen, 

where Ppg is the parallel transport along the geodesic joining p to q. 

Proposition 2.5. Let Q G M he an open convex set and / : M — )■ M a twice continuously 
differentiahle function on Vt. If Hess pf is hounded on Q, then the gradient vector field grad f 
is Lipschitz on Q. 

Proof. The proof is an immediate consequence of the fundamental theorem of calculus for 
vector fields, see for example [TT]. □ 
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3 Directional derivatives 



In this section we present some properties of the directional derivative of a convex function 
defined on a Hadamard manifold, including a characterization of the directional derivative 
and of the subdifferential of the maximum of a certain class of convex functions. We also 
give a definition of the generalized directional derivative of a locally Lipschitz function (not 
necessarily convex), see Azagra et.all [3], and an important property of the subdifferential 
of the maximum of continuously differentiable functions. 



3.1 Directional derivatives of convex functions 

In this subsection we present the definition of the directional derivative of a convex function 
defined on a Hadamard manifold and some properties involving its subdifferential, which 
allow us to obtain an important property of the subdifferential of the maximum of a certain 
class of convex functions. 

Let C M be an open convex set and / : M — > i? a convex function on Q. Take p E Q, 
V G TpM and 6 > and let 7 : [—6 , 5] — )■ f2 be the geodesic segment such that 7(0) = p and 
7'(0) = V. Due to the convexity of / o 7 : [—S , 5] — )■ M, the function : (0 , 5] — )■ M, given 

is nondecreasing. Moreover, since / is locally Lipschitzian, it follows that is bounded near 
zero. This leads to the following definition: 

Definition 3.1. Let Q G M be an open convex set and f : M ^ R a convex function on VL. 
Then the directional derivative of f at p E Q in the direction of v E TpM is defined by 

f{p,v) := Mm q^{t) = jnf (8) 

where S > and 7 : [—6 , S] ^ Q is the geodesic segment such that 7(0) = p and 7'(0) = v. 

Proposition 3.1. Let Q G M be an open convex set and / : M — M a convex function on 
Q. Then, for each fixed p G Q, the subdifferential df{p) is convex. 

Proof. See Theorem 4.6 of [32i|, p. 74. □ 

Proposition 3.2. Let Q G M be an open convex set and / : M — > M a convex function on 
Q. Then, for each point fixed p G Q, the following statement holds: 

f'{p,v) = max,ga/(p)(s,f), for all v G TpM; 
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ii) df{p) = {se TpM : f{p,v) >{s,v),ve T^M}. 
Proof. See [8]. □ 

Proposition 3.3. Let T be a compact set, Q G M an open convex set and /i : M x T — )■ M 

a continuous function on Q x T such that , r) : M — )■ M zs convex on Q for all t eT . If 
f : M ^ M. is given by f{p) = max^gy h{p, r), then f is convex on and 

f'{p, v) = max^er(p) h'{p, t,v), p G fi, v E TpM, 

where T{p) = {r G T : f{p) = h{p,T)}. Moreover, if h{.,T) is difjerentiable on Q for all 
T E T and grad^ h{p, .) is continuous for all p E Q, then: 

df{p) = conv {gradp h{p, r) : r E . 

Proof. Since T is compact / is well defined and its convexity follows from Proposition 12.41 
Now, take p E Q, v E TpM and the geodesic segment 7 : [—6, 6] ^ Q, 6 > 0, such that 
7(0) = p and 7'(0) = v. Using that T is compact we have T{p) ^ 0. Hence, taking r G T{p) 
we obtain from definition of / and T{p) 

fm) - f{p) . M7(t),r)-%,r) 

> , tE{0,d). 

Given that / and h{., r) are convex, letting t goes to 0, the above inequality yields: 

f'{p,v)>h'{p,r,v), pEn,vETpM, r ET{p). 

Therefore, 

f'{p,v)> snp h\p,T,v), pEVL.vETpM. (9) 

rgr(p) 

Now, we are going to prove the equality in the above equation. Let {tfc} C (0, 5) such that 
tk converges to as /c goes to +00. We define 

/:=7(4), GT(/). (10) 

The last equality implies that f^Clitk)) = h{'y(tk),Tk). Therefore, as / is convex, combining 
([8]) and the definition of /, we obtain: 

y/j^^ ^ filjtk)) - f{p) ^ h{-f{tk),Tk) - h{p,Tk) 
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Since {rk} C T and T is compact, we can suppose (taking a subsequence, if necessary) that 
it converges to f G T as A; goes to +00. Thus, letting k goes to +00 in the latter inequality, 
we have: 

f{p,v) < h'{p,f,v), 

Since h is continuous and x h- )■ h{x, r) is convex for each t & T. Note that, if f G T[p), then 
the last inequality implies that (Q holds with equality. So, in order to prove the first part it 
is sufficient to prove that f G T{p). First note that using ffTOj) and the definitions of T{p'^) 
and /, we conclude that: 

h{p\ Tk) = max,6T h{p\ r) > h{p\ r), r G T. 

Hence, letting k goes to +00, we have h{p, f) > h{p, r), for all t E T, which, combined with 
the definition of /, give f{p) = h{p,f), and the first part is completed. 

To prove the second part, take p E fl and r G T{p). From Proposition 12. the convexity 
of h{., t) implies: 

h{q, r) > h{p, r) + (gradp h{p, r), exp^ ^ q), q e n. 

Because r G T{p) we have h{p,T) = f{p), which along with the definition of / as well as 
with latter equation yields: 

fil) > f{p) + (gradp h{p, r), exp^ ^ q). 

So, gradp /i(p,r) G df{p). Since Proposition 13.11 implies that df{p) is convex, we conclude 
that: 

convjgradp r) : r G T{p)} C dfijp). 
We claim that this inclusion holds with equality. Indeed, assume by contradiction that: 

3y edf{p), y i conv{gradp h{p, r) : t e T{p)}. 

Due the fact that gradph^p, .) is continuous and T{p) is a compact set we conclude that the 
set convjgradp r) : r G T{p)} is compact. Thus, by the Separation Theorem on TpM, 
there exists v G TpM — {0} and a G M such that 

{y,v) > a> {gradp h{p,T),v), Vr G T{p). 

Since h'{p, r, v) = (gradp h{p, r), f )), it follows from the latter inequality and first part of the 
proposition that 

(y, v) > max^gT(p) h'{p, r, v) = f'{p, v). 

Since y G df{p), we obtain a contradiction with Proposition 13.21 i. This proves our claim 
and conclude the proof. □ 



10 



Corollary 3.1. Let Q G M be a open convex set and hi : M ^ M. a differentiable convex 
function on Q for i E I \= {1, m}. If h : M is defined by h{p) := maxjg/ hi{p), then: 

dh{p) = conv{grad /ij : i G I{p)} = <y e TpM : y = ^ grad /ii (p) , ^ a, = 1, a, > > , 

[ «e/(p) iei{p) ) 

where I{p) :={«€/: h{p) = hi{p)}. In particular, p minimizes h on Q, if and only if, there 
exist ca > , i E I (p) , such that: 

= ^ aigYadhi{p), ^ = 1. 

iG/(p) «e/(p) 

Proof. It follows directly from Proposition 13.31 □ 

3.2 Directional derivatives of locally Lipschitz functions 

In the sequel we present the definition of generalized directional derivative of a locally Lip- 
schitz function (not necessarily convex) and an important property of the subdifferential of 
the maximum of continuously differentiable functions. 

Definition 3.2. Let Q G M be an open convex set and / : M — > M a locally Lipschitz 
function on Q. The generalized directional derivative of f at p G Q in the direction v G TpM 
is defined by: 

roi ^ /(expgt(Dexpp) 

/ := limsup , (11) 

where (-Dexpp)g^p-i^ denotes the differential of exp ^ atexp^^q. 

It is worth noting that an equivalent definition has appeared in [3]. 

Remark 3.1. The generalized directional derivative is well defined. Indeed, let Lp > be 
the Lipschitz constant of f in p and 6 = 6{p) > such that 

|/(expgt(L)expp)^^p-i^w) - /(g) I < Lpd{expgt{D expp)^^^-i ^v, q), q G Bs{p), t G [0,6). 
Since d{expgt{D expp)^^^-! ^v, q) = t\\{D expp)^^^-! ^v\\, the above inequality becomes: 

\f{exp^t{Dexpp)^^^-igV) - f{q)\ < Lpt\\{D exp^)^^^-i ^vl q G Bs{p), t G [0,6). 
Since limg_^p {D expp)^^^-i = v, our statement follows from the latter inequality. 
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Remark 3.2. Note that, if M = W^ then exp^w = p + w and 

^(exPp)exp-ig^ = ^- 

In this case, (fTTj) becomes: 

fUP,'") = limsup — ^M^ 

which is the Clarke's generalized directional derivative in the Euclidean case, see There- 
fore, the generalized differential derivative on Hadamard manifolds is a natural extension of 
the Clarke's generalized differential derivative. 

Next we generalize the definition of subdifferential for locally Lipscliitz functions defined 
on Hadamard manifolds, see Proposition 13.21 item ii. 

Definition 3.3. Let Vl d M he an open convex set and f : M ^ M. a locally Lipschitz 
function on VL. The generalized subdifferential of f at p & fl, denoted by d°f{p), is defined 
by: 

d°f{p) := {w E TpM : f°{p,v) > {w,v) for all v G TpM}. 

Remark 3.3. // / is convex on Q, then f°{p,v) = f'{p,v) (respectively, d°f{p) = df{p)) 
for all p G Q, i.e., the directional derivatives (respectively, subdifferential) for Lipschitz 
functions is a generalization of the directional derivatives (respectively, subdifferential) for 
convex functions. See Claim 5.4 in the proof of Theorem 5.3. 

Definition 3.4. Let Q G M be an open convex set and / : M — )■ M a locally Lipschitz 
function on Q. A point p E Q is a stationary point of f if E d°f{p). 

Lemma 3.1. Let Q G M be an open set. // / : M — )■ M zs locally Lipschitz on Q and 
(7 : M — ?► M «s continuously differentiable on Vt, then: 

{f + 9np,v)=np,v) + g'ip,v) pen, veT,M. (12) 

As a consequence, 

d\f + 9){P) = d°f{p) + grad p E n. (13) 

Proof. Using the definition of the generalized directional derivative and simple algebraic 
manipulations, we obtain: 

/(exp^t(Dexpp)^^p-i^i;) - /(g) ^(exp^t(Dexpp)^^p-ig-;;) - g{q) 
t ^ t 



U+gYip^v) = limsup 
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From upper limit properties together with the definition of the generahzed directional deriva- 
tive and Remark I3.3[ we obtain 

{f + 9np,v)<np,v) + g'ip,v). (14) 

Because f°{p,v) = ((/ + g) + i—g))° {p,v), above inequality implies in particular that 

np,v)<if + gnp,v) + i-gy{p,v), 

which is equivalent to 

{f + 9np,v)>np,v) + g'{p,v). 

So, last inequality together with inequality f|T4l) yields the equality f|T2|) . 

The proof of the equality f|T3l) is an immediate consequence of the equality f|T2|) and the 
definition of the generalized subdifferential. □ 

It is possible to prove that the next result holds with equality. However we will prove 
just the inclusion needed to prove our main result. 

Proposition 3.4. Let Q G M be an open convex set and / = {1, ...,m}. Let /j : M — )■ M &e 

a continuously differentiable function on Q for all i G I and / : M — )• M defined by 

f{p):= maxig//i(p). 
Then f is Lipschitz locally on Q and for each p E Q 

conv{grad/i(p) : i e I{p)} C d° f{p), 
where I{p) := {i e I : fi{p) = f{p)}. 

Proof. Since /j is continuously differentiable on Q we conclude that fi is Lipschitz locally in 
Q, for all i G I. Thus, for each p & Q and i & I, there exists Si, Li > such that 

\fi{p) - fi{(l)\ < Lid{p,q), p,qe BsXp)- 

On the other hand, 

I maxig/ f^{p) - maxig/ fi{q)\ < max^g/ \fi{p) - fi{q)\. 
Combining two last equations with the definition of /, we obtain: 

\fip)- fiq)\<Ld{p,q) p,qeBs{p), 
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where 5 — minimi 5i and L — maxjg/Lj, which prove the first part. 

In order to prove the second part, take p & Q, u E conv{grad/i(p) : i e /(p)} and 
V e TpM. Then, there exist a constant ctj > for i e I{p) with X]ie/(p) — ^ such that 

w = ^ aigrad/i(p). 
iei{p) 

Since /j is differentiable for all i e /, simple algebraic manipulation yields 

{u,v) = ^ ai{gradfi{p),v) = ^ aif[{p,v). 

As / is locally Lipschitz at p, the definitions of /, I{p) and generalized directional derivative 
imply: 

n{p,v)<np,v), 

which, together with the latter equation, gives {u,v) < f°{p,v), and the proof follows from 
the definition of d°f{p). □ 



4 Proximal Point Method for Nonconvex Problems 

In this section we present an application of the proximal point method to minimize a real- 
valued function (not necessarily convex) given by the maximum of a certain class of contin- 
uously differentiable functions. Our goal is to prove the following theorem: 

Theorem 4.1. Let Q C M be an open convex set, q E M and I — {1, m}. Let : M — >■ R 

be a continuously differentiable function on Q and continuous on ( closure of Q ), for all 
i & I, and / : M ^ M defined by 

f(p) := maxigj/j(p). 

Assume that — oo < inip^M fip), grad/j is Lipschitz on D, with constant Li for each i & I 
and 

Lfifiq)) ^{peM: f{p) < f{q)} C inf /(p) < f{q). 

Take < A and a sequence {Afe} satisfying maxjg/Li < < A and p e Lf{f{q)). Then the 
proximal point method 

:= argmin^^M + ^d\p,p^)'^ , A; = 0, 1, ... , (15) 

with starting point p*^ — p is well defined, the generated sequence {p*^} rest in Lf{f{q)) and 
satisfies only one of the following statements 
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i) {p''} is finite, i.e., p^^^ = for some k and, in this case, p^ is a stationary point of 
f, 

a) {p''} is infinite and, in this case, any accumulation point of {p^} is a stationary point 
off. 

Moreover, assume that the minimizer set of f is non-empty, i. e., 
hi) f/* = {p:/(p)=infpgM/(p)}7^0. 

Let c G {inf M f{p), f{(l))- If, in addition, the following assumptions hold: 
h2) Lf{c) is convex and f is convex on Lf{c); 

h3) For all p G Lf{f{q)) \ Lf{c) and y{p) G d° f{p) we have \\y{p)\\ > 5 > 0, 
then the sequence {p^} generated by (ITS!) with 

maxjg/ Lj < Afc < A, A; = 0, 1, . . . (16) 
converge to a point p* eU* . 

Remark 4.1. The continuity of each function fi on Vt in h2 guarantees that the level sets 
of the fuction f , in particular the solution set U* , are closed in the topology of the manifold. 

In the next remark we show that if VL is bounded and is convex on fi, for aU i & I, 
then / satisfies the assumptions h2 and h3. 



Remark 4.2. If fi is also a convex function on Q for each i E I then by the Proposition 2.4 



the function f is convex on and the assumption h2 is satisfied for all c < f{q). Moreover, 
from Remark \3.3\ 

df%p) = dfip), Wpen. (17) 

Take c G (mfp^Mfip), /(o')) o,nd let us suppose that hi hold and fl is a bounded set. Then, 
we have 

< sup {dip*,p) ■.p*eU*,pe Lfifiq)) \ Lfic)} = e < +00. (18) 

Let p* G U* be fixed, p G Lf{f{q))\Lf{c) andy{p) G df{p). The convexity of f on Q implies 
that: 

{y{p) , -expp^p*) > f{p) - f{p*). 

Since \\y{p)\\\\exp-^p*\\ > {y{p), - exp'^ p*) , d{p\p) = ||exp-ip*||, p G Lf{f{q))\Lf{c) 
and U* is a proper subset of Lf{c), from the above inequality, we obtain: 

\\yip)\\d{p*,p) >c-f{p*) >0. 
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Thus, from ( 1T8|1 and latter inequality 

\\y{p)\\e>c-fip*)>0. 
Therefore, choosing 6 = {c — f{p*))/e, we have: 

\\y{p)\\>S>0, 

which, combined with (fTTI) . shows that f satisfies h3. 

In order to prove proving the above theorem we need of some prehminary results. From 
now on we assume that all assumptions on Theorem 14. II holds, with the exception of hi, h2 
and h3, which will be assumed to hold only when explicitly stated. 

Lemma 4.1. For all p & M and A satisfying 

sup Li < A, 

the function fi + (A/2)(i^(. ,p) is strongly convex in Q with constant A — sup^g^Lj. Conse- 
quently, f + (A/2)(i^(. ,p) is strongly convex on Q with constant A — supjgj-Lj. 

Proof. Due to the finiteness of J, the function / is well defined. Take i & I, p & M and 
define hi := + (A/2)(i^(. ,p). Note that grad/ij(p) = grad/j(j9) — Xexp~^p. Thus, for all 

(^gp^grad/ii(p) -grad/ii(g), exp'^) = (^gp^ grad /j (p) -grad/i(g), exp'^ p) 

- KPqp expp ^ p - exp-^ p, exp-^ p) . 

Since (P"^ grad /^(p) -grad /^(g), exp^^) > -||Pgpi grad /^(p) -grad || exp^ VII, using 
equality d{p,q) = \\ exp~V||, Proposition 12.21 and the above equation, we obtain: 

(^<?p^grad/ii(p) - grad/ii(g),expg V) > "ll^gp^ grad/i(p) - grad/i(g)||c/(g,p) + \d^{p,q). 

Now, as grad fi is Lipschitz on Vt with constant Li and the parallel transport is an isometry, 
the latter equation becomes 

(^gp^grad/ii(p) - grad/ii(g),exp^ V) > {\ - Li)d^{p,q). 

By hypothesis A > supjg/Lj. Hence, the above equation and Definition 12.11 imply that 
grad/ij is strongly monotone with constant A — supj^jLj. Therefore, from Proposition 12.31 
we conclude that hi is strongly convex with constant A — supjgjLj. It easy to see that 

maxjg//ii = / + {\/2)d^{. ,p). 

Thus using Proposition 12.41 the proposition follows. □ 
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Corollary 4.1. The proximal point method (flSj) applied to f with starting point p'^ = p is 
well defined. 

Proof. Assume that p^ G Lf{f{q)) for some k. Note that the minimizers of ipk '■= f + 
(Afc/2)(i^(.,p'^), in case they exist, are in L^^{ipk{p'')) C Lf{f{q)) C Q, more precisely, 

argminpg^^?/'fe(p) = argminpgi^j^^,(pfe)) Vfc(p)). 

As / is continuous on Q, L^^{ipk{p^)) is closed in the topology of the manifold M. Moreover, 
since sup^gj. Lr < Xk, we conclude from Lemma I^TTl that the application ipk is strongly convex 
on Q with constant /3 = A — sup^g^ ^r- In this conditions, for ipk to has an unique minimizing 
at M is sufficient that L^^[ipk{p^)) is a bounded set. From the convexity of ipk on ^ we have, 
in particular, that L^^{%ljk{p^)) is a convex set. Assume by contradiction that L psif.{ psikip'^)) 
is unbounded. Then, there exists v G TpkM and an a geodesic 7 : [0, +00] — L^i_{ipk{p^)) 
such that 7(0) = p'^ and 7'(0) = v. Take to > 0. For all t > to, we have 

Now, taking into account that ipk is strongly convex with constant /3, from the last equality 
follows that 

Miito)) < jMiit)) + (1 - j)Mim - 1(1 - j)jP\\vrt'. 

Because, ipk{'yit)),tpk{liO)) G L^^{ipk{p''), from the last inequality, we obtain 

Mlito)) < Mp") - lit - to)to(3\\vf, 

which is a contradiction, since ipkilito)) is finite and ipk{p^) — ^{t — to)to(3\\v\\'^ goes to —00 
as t goes to +00. Therefore, L^^^ipkip^)) is bounded and, consequently, p^~^^ is well defined. 
Since p^ = p E Lf{f{q)), the proof follows from a simple induction argument. □ 

Lemma 4.2. Let {p''} be the sequence generated by the proximal point method ( IT5l) . Then 
the following statements holds: 

+ (X,/2)d'{p>^+\p') < /(/•), A; = 0, 1, . . .; 

u) {/} C Lfifiq)); 

III) G 9 (/ + ^d'^{. , /)) A; = 0, 1, . . .; 
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iv) -oo < / = limfc->oo f{p^); 

v) \imk^ood{p^^^,p^) = 0. 

Proof. The first item is an immediate consequence of (fT5l) . wliicli implies tliat {f{p'')} is 
monotonous and nonincreasing of where follows the item ii. Since maxjg/ Li < X^, Lemma l^?T] 
implies / + {Xk/2)(P[. ,p^) convex on fl, which, together with f|T5l) . proof item iii. Using that 
{JXp'')} is monotonous nonincreasing and that — oo < inf p(z m f{p), item iv follows. Finally, 
item V is a consequence of items i and iv. □ 

Lemma 4.3. Let {p''} be the sequence generated by the proximal point method ( |T5l) with Xk 
satisfying (fT6l) . Assume that hi and h2 holds. If p^ G Lf{c) for some k then {p'^} converges 
to a point p* e U* C Q. 

Proof. By hypotheses, p'' G Lf{c) for some k, i.e., there exists ko such that f{p''°) < c. Then, 
from Lemma [4.21 item i, {p^} C Lf{c) for all k > k^. On the other hand, from (fT6l) we have 



Therefore, using hi and h2 the result follows from similar arguments used in the proof of 
Theorem 6.1 of [13j. □ 

Lemma 4.4. Let {p''} be the sequence generated by the proximal point method ( fTSi) with Xk 
satisfying ( |T6l) . // h3 holds then after a finite number of steps the proximal iterates go into 
the set Lf{c). 

Proof. First note that since vaip^M f{p) < c we have Lf{c) ^ 0. Suppose by contradiction 
that p'' G Lf{f{q)) \ Lf{c) for all k. From (fT6ll we have max^g/ Lj < A^. Since (l/2)(i^(.,p) 
is a different iable function with 



applying Lemma Wl] with A = A^, Lemma [3TT] with g = (l/2)(i^(.,p) and the first part of 
Proposition 13.41 we obtain: 



Hence, it is easily concluded from the last equality and item iii of Lemma [4.21 that: 




grad((l/2)ci^(g,p)) = -exp/p 




= 0,1, 
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Since p^^^ E Lf{f{q)) \ Lf{c), assumption h3 and the latter equation give 

II Afc exp^fcVi /II > ^, k = 0,l, 

From ( IT6i) we have < A. As d{p^^p'^^^) = \\ exp~fe+ip^|| the last inequality implies that 

%^/+^)>£, A; = 0,1,.... 

So, from item v of Lemma 14.21 we arrive at a contradiction. Therefore there exists ko such 
that f{p^°) < c and the result follows from the item i of Lemma [4.21 □ 

Proof of Theorem 14.11 

Proof. The well definition of the proximal point method follows from the Corollary 14. 1[ Let 
{p*^} be the sequence generated by the proximal point method. As p° = p G item i 

of Lemma [4.21 implies that the whole sequence lies in Lf[f[q)). From item iii of Lemma [4. 2 ^ 
we have: 

0G9(^/ + ^rf^(.,/))(/+^), fc = 0,l,.... 

Since maxig/Lj < A^, Lemma WA\ implies that fi + (Afc/2)(i^(. ,p'') and / + {\k/2)(f{. ,p^) 
are strongly convex. Thus, applying Corollary 13.11 with hi = fi + {Xk/2)cP{. ,p^) and h = 
f + (Afc/2)(i^(. we conclude that there exists constant af^^ > with i E I[p'^^^) such 
that 

ig7(pfe+i) \ ^ jei'(p'=+i) 

This tells us that 

0= «^'grad/,(p'=+l)-A,exp^-i.p^ ^ = 1' A; = 0, 1, . . . . (19) 

If the sequence {p^} is finite then there exists k such that p^^^ = p^. In this case, exp~j.^_|_i p^ = 
and the first equality in (fT9ll becomes: 

0= J2 «^'grad/,(/+^), 

jg/(pfc+i) 

which, together with Proposition 13. 4[ implies that G d°f{p^). Hence, p^ is a stationary 
point of /. 
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Now, assume that sequence {p^} is infinite and p is an accumulation point of it. Let 
{a'l^^} C be tfie sequence defined by 

Since ^jg/(pfc+i) af^^ = 1 we have = 1 for all A;, where || ||i denotes the sum norm 

in M™. Thus {a'^"'"^} is bounded. Let and {a''"'^^} be the subsequence of {p''^^} 

and {a'^'*"^}, respectively, such that lims^+ooP^°^^ = P and lims_j.+oo ct'^^'^"'^ = a. As / is 
continuous on Q, item ii of Lemma [4.21 implies that p G Lf{f{q)) C Since / is finite we 
can assume without loss of generality that 

/(p'^i+i) = /(p'^^+i) = = (20) 

and equation (fT9|) becomes 

= 5Z grad /.(p'^^+i) - Afc, exp;,i p'^ , ^ = 1, s = 1, 2, . . . . 

On the other hand, item v of Lemma 14.21 implies 

lim c/(/=+\/0 = lim || exp'/^+i || = 0. 

s—^oo s—^oo t' 

As Afc^ is bounded, lim^.j^+oop'^"'''^ = P and lims_>+oo a^''"*'^ = a, letting s goes to +oo in the 
above equality, we conclude 

= ^aigrad/,(p), ^^^ = 1. 

Using definition of /(p), equation ( l20l) and the continuity of /, we obtain / C I{p)- Therefore, 
as p G fi, it follows from the Proposition 13.41 that 

G d°f{p), 

i.e., p is a stationary point of /, which concludes the proof of the first part of the theorem. 
The second part follows from Lemma 14.31 and Lemma 14. 4[ □ 

5 Examples 

In this section we present two examples. In the first we consider a non-convex minimization 
problem where the objective function is defined on a Hadamard manifold with curvature 
identically zero. In the next example we "generalize" this example to one where the curvature 
of the Hadamard manifold is not identically zero. In both examples, the classical local 
proximal point method (see [16]) as well as the Riemannian proximal point method (see 
[13] ) does not apply. However, the method proposed in this paper applies. 
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5.1 Example 

Let (ffi++, ( , )) be the Riemannian manifold, where = {x G M : x > 0} and ( , ) is the 
Riemannian metric {u,v) = g{x)uv with g : M++ — ?■ (0,+oo). The Christoffel symbol and 
the geodesic equation are given by 

respectively. Moreover, in relation to the twice differentiable function h : IR++ — ?■ M, the 
Gradient and the Hessian of h are given by 

grad/i = g~^h', hess h = h" — Th', 

respectively, where h' and h" denote the first and second derivatives of h in the Euclidean 
sense. For more details see [32]. In the particular case of g{x) = x~^, 

T{x) = —x^^, gradh{x) = x'^h'{x), hess h{x) = h" (x) + x^^ h' (x) . (21) 

Moreover, the map : M — > M++ defined by (p{x) = e^' is an isometry between the Euclidean 
space M and the manifold ( , )) and the Riemannian distance d : x M++ — )■ M+ is 

given by 

d{x,y) = \cp-\x)-if-\y)\ = \\n^l (22) 

see, for example [9]. Therefore, (IR++, ( , )) is a Hadamard manifold and the unique geodesic 
X : M — )■ R_|__)_ with initial conditions x(0) = Xq and a;'(0) = f is given by 

x{t) = xoe(^/^'°)*. 

From the above equation it is easily seen that any interval / C is a convex set of the 
manifold (M++, (, )). 

Let /i, /2, / : — )• M respectively be given by 

fi{x) = \n{x), /2(x) = - ln(x) + e"^"^ - e"^ f{x) = maxj=i,2 fjix), 

and consider the problem 

min/(a;) , . 

s.t. X e M++. ^ ^ 

Take a sequence {A^} satisfying < A^. From (!22|) . the proximal point method (fT5l) becomes 
x^+i := argmin,,^^^ |/(x) + y (^) } , A; = 0, 1, ... . 
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Note that — oo < infa;gK^^ /(x) = and, being /i and /2 twice differentiable functions on 
the last expression in f l2T|) imphes that 

2 

hess = and hess /2(x) = (4 - -)e-^'^, x e M++. (24) 

Let < e < 1/4, g = 5/16 and Q = (e, +oo). So, = inf^gK^,^ f{x) < f{q) and Lf{f{q)) C il. 
Moreover, hess /i, hess /2 are bounded on Q and consequently grad/i,grad/2 are Lipschitz 
on Q. We denote by Li the constant of Lipschitz of grad/j, i = 1, 2. Clearly the assumption 
hi of the Theorem 14.11 is verified with U* = {1}. 

We claim that there exists c G (0, /(1/2)) = (0, /i(l/2)) such that Lf{c) is convex and / 
is convex on Lf{c) (in the Riemannian sense). Indeed, as hess /i > in M++ and hess /2 > 
in [1/2, +oo). Theorem 6.2 of [9] implies that /i is convex on M++ and /2 is convex on 
[1/2, +oo). Thus, it follows from Proposition 12.41 that / is convex on [1/2, +oo). Note that 
for all c G (0, /(1/2)), we have Lf{c) fl [1/2, +oo) = Lf{c). Hence, from the convexity of / 
on (1/2, +oo) we conclude that Lf{c) is convex, which proves the claim. So, Lf{c) and / 
satisfy assumption h2 of Theorem 14. H for example with c = /(3/4). 

Now, note that Lj{f{q))\Lj{c) = [5/16,3/4) U (a, 6], where a = (4/3)e(^"'^'-^"') and 
b = (16/5)e*-'^ ^/*-e Moreover, / is differentiable on Lf(f(q))\Lf(c) with grad/(x) = 
grad/i(x) for x G [a,b] and grad/(x) = grad/2(x) for x G [5/16,3/4]. From the second 
expression in (ETj) . we have: 

grad/(x) = X, x G (a, 6] and grad/(x) = —x — 2x^e~^^, x G [5/16,3/4]. 

Thus, we have ||grad/(x)|| > || grad /(5/16) || > 2/5 and / satisfies the assumption h3 of 
Theorem 14.11 

Summarizing, all assumptions of Theorem 4.1 are satisfied with Q = (e, +oo), q = 5/16, 
c = /(3/4) and 5 = 2/5. Therefore, letting x° G M++ and A > such that x° G Lf{f{q)) 
and maXjg/Lj < < A, the proximal point method, may be applied for solving the above 
nonconvex problem. 

Remark 5.1. Function f{x) = max{ln(x), — ln(x) + e~^^ — e~^}, in the above example, is 
nonconvex (in the Euclidean sense) when restricted to any open neighborhood containing its 
minimizer X* = 1. Therefore, the classical local proximal point method (see JJBjl) cannot be 
applied to minimize this function. Also, as f is nonconvex in the Riemannian sense as well, 
the Riemannian proximal point method (see fT3f ) cannot be applied to minimize this function 
either. 
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5.2 Example 

Let S" be the set of the symmetric matrices, be the cone of the symmetric positive semi- 
definite matrices and ^_ be the cone of the symmetric positive definite matrices both nxn. 
For X,YeSl,YyX{orX^ Y) means that y - X G §™ and F ^ X (or X ^ F) means 
that Y — X E We will denote the Frobenius norm by || . \\f- 

Following Rothaus p8], let M := (§+_|_, (, )) be the Riemannian manifold endowed with 
the Riemannian metric induced by the Euclidean Hessian of ^'(X) = — IndetX, 

{U, V) = tT{V^"{X)U) = tT{VX-^UX-^), X G M, U,V e TxM, (25) 

where tr(A) denotes the trace of matrix A G S" and TxM ^ S", with the corresponding 
norm denoted by || . ||. In this case the unique geodesic segment connecting any X, F G M 
is given by 

7(t) = xi/2 (x-i/2rx-i/2)* XV2, t G [0, 1], 

see, for instance, [23]. More precisely, M is a Hadamard manifold, see for example [17], 
Theorem 1.2. page 325. From above equality it is immediate that 

7'(0) = X'/Hn {X-'/^YX-'/^) X^l\ 

Thus, for each X G M, exp^^"*^ : M — )■ TxM and expjs^ : TxM — )■ M are given, respectively, 
by 

exp3,i Y = Xi/2 In [X'^I^Y X'^'^) X^l\ exp^ V = x^l^^""-"'"^ X^l\ (26) 

Now, since the Riemannain distance d is given by (i(X , Y) = \\expx^Y\\, from (l25l) along 
with first expression in fl26l) . we conclude that: 

n 

d\X, Y) = tr (In^ X-'/^YX~'/^) = ^ In^ A, (x-^FX-^) , (27) 

i=l 

where Xi{X~^YX^^) denotes the i^^ eigenvalue of the matrix X~^YX~^ . The gradient and 
the Hessian of a twice different iable function F : — M is given, respectively, by: 

gradF(X) = XF'(X)X, hess F(X) (V, = tr (rF"(X)V) +tr (F'(X)VX~V) , V G TxM, 

where F'{X) and F"{X) are the Euclidean gradient and Hessian, respectively. We remind 
that a twice differentiable function F : — )■ R is convex on the manifold M if it satisfies 
the condition: 

hess F(X) (V, V^) > 0, X e M, V e TxM. 
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Let Fi, F2, F3 : — )► M be given, respectively, by: 

= IndetX, F2(X) = -41ndetX + e'^*"^^ - 6"^", F^iX) = ti - n. 

Note that Fi, F2 and F3 are twice differentiable functions on + . Taking X G TxM and 
G TxM, the Euchdean Gradients are given by: 

F[{X) = X-\ F^{X) = - 2e-2*'"^, F^(X) = -X^^ 

and the Euchdean Hessians are given by: 

F['{X)V = -X-WX-\ 

F^{X)V = AX-^X-^ + Ae-^^'^V, 

F.;^{x)v = + x-^vx-\ 

Thus, the Riemannian gradients of Fi, F2 and F3 are given by: 

gradFi(X) = X, gradF2(X) = -4X - 2e-^''^X^, gradF3(X) = -J, (28) 

where / denotes the identity matrix, and Riemannian Hessians of Fi,F2 and F3 are given 
by: 

hessFi(X) {V,V) = 0, 

hessF2{X){V,V) = 2e'^''''tT [{21 - X-^)V''] , (29) 
hessF3(X) {V,V) = tr (X^VX^V) = HX" VX-^/2|||. 

It is easily seen that the functions Fi and F3 are convex everywhere and the function F2 is 
convex on any convex subset of the set 

C := {X G : 21 - X'^ ^ O} = {X G : Amin(^) > 1/2} , (30) 

where Amin(^) denotes the minimum eigenvalue of the matrix A. 
Let F : ^ M be given by 

F(X) =max,=i,2,3Fi(X). 

From Proposition 12.41 F is convex on any convex subset of C. Consider the following opti- 
mization problem 

minF(X) , . 

s.t. X G ^ ^ 
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If {rjk} is a sequence satisfying < rjk, then from (|27|1 the proximal point method (|T5l) 
becomes 

= argmin^,s;, |/W + f E^^'^^ (x-^X,X-^)| . 
Define the sets f/i, and U3 as 

f/i := {X G : detX = 1}, U2 := {X G : det X > 1}, f/3 := {X G S'^^ : detX < 1}, 
and note that S*^^ = f/i U t/2 U t/3 and t/^ H Uj = 0, for j = 1, 2, 3, 2 ^ j. 
Claim 5.1. 

[ i^3(X) > 0, X G t/i; 

F(a;) = I max{Fi(X),F3(X)} > 0, X G U2; (32) 
[max{F2(X),F3(X)}>0, X G t/3. 

/n particular, inf§n_^ F{X) = and U* = {/}, where U* is the solution set of the problem fl?I|) 
and / is the n x n identity matrix. 

Proof. First of all, note that for each X G + , we have 

^'d^<^. (33) 

n 

If X G f/i then detX = 1. Using the above inequality we obtain trX > n which implies 

F2(X) = e-''''^ - < = Fi(X). 

On the other hand detX~^ = 1. Hence, (133|) gives us trX~^ — n > and thus -F3(X) > 0. 
So, from definition of F and taking into account the above inequality, we conclude 

F(X) = F3(X) > 0, X G U,. (34) 

If X G t/2 then detX > 1. Thus, from (133!) we also obtain trX > n. Hence, 

F2(X) = -41ndetX + e~2*''^ < -IndetX < 0. 

Since, detX > 1 we have -Fi(X) = IndetX > 0. Therefore, 

F(X) = max{Fi(X), F3(X)} > 0, X G U2. (35) 
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Finally, if X G f/3 then detX~^ > 1. Hence, inequality fl33|) implies that trX~^ > n and 
consequently F^{X) = tiX^^ - n > 0. As detX < 1 we have -Fi(X) = Indet(X) < 0. 
Thus, 

F{X) = max{F2(X), F^{X)} > 0, X G f/3, (36) 

and the the first part of the claim is proved. 

From the first part we conclude that F{X) > 0, for all X G Since F{I) = we have 
inf§n^ F{X) = 0. In order to prove the last statement, note that if F{X) = then X G ?7i. 
Definition of Ui and ([32]) give us detX"^ = 1 and F(X) = tr(X~i) - n for all X G Ui. 
Hence, if X G f/* then tr(X~^) = n and detX^^ = 1, which implies that ([33D holds with 
equality. On the other hand, fl55]) holds with equality if only if Ai(X) = ... = A„(X) > 0. 
As tr(X~^) = n we conclude that Aj(X) = 1 for i = 1, . . . ,n and the result follows. □ 

Claim 5.2. Fi(X) > F3(X) for X G A, where 

A := {X G : X - / ^ 0} = {X G §!^+ : X^UX) > l} C U2. 

Consequently, F{X) = Fi(X) for X e A. 

Proof. Define the following function 

^|J{X) :=Fi(X)-F3(X). 

From the Mean Value Theorem on the Euclidean space there exists X G f/2 such that 

^(X) = V'(/) + tr (V^'(X)(X - /)) = tr ((X-i + X'')iX - /)) . 

If X G A then X ~ I e Sl^. Since X - / and X'^ + X'^ belong to and the trace 
of the product of positive definite matrices is positive, we conclude from the above equality 
that ipiX) = Fi{X) - FsiX) > for all X e A, which proof the first part of the claim. The 
second part of the claim follows from Claim 15.11 . □ 

Claim 5.3. F(X) = F2(X) for X eB, where 

5 := {X G : (1/4)/ ^ X ^ /} = {X G : 1/4 < A^i„(X), A„,ax(X) < l} . 

Proof. Note that 5 C f/3 n {X G : trX < n]. So, from ([361), to prove this claim it 
suffices to verify that -F2(X) > -F3(X) for all X e B. Let : M++ — )■ M be the function given 
(j){t) := — 41nt — 1/t + 1. From the convexity of —Alnt, we have 

0(t) > -41nl-4(t-l)-- + l >0, tG (1/4,1). (37) 
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On the other hand, definitions of F2, F4 and yields: 

n 

F,{X) - F,{X) = - + J] 0(A,(X)). 

i=i 

Since e"^*''"^ — e~^" > for X E B, combining ( 137|) with the latter equality we conclude 
that F2{X) > -F3(X) for X E B. Hence, the desired equality follows. □ 

It is easy to verify that F is a coercive function. Thus, Lp{a) is bounded for all a G M, 
i.e., all the level sets of F are bounded. Take 

g:=diag(l/4,...,l/4). (38) 

As Lf{F{Q)) C is a bounded set, then there exists k > A sufficiently large so that 

A^in(X) > !//€, A^ax(X)<ft:, XeLpiFiQ)). (39) 

Now we define 

n:={X e : trX"^ < kti}. 

Since the function 3 X trX~^ is convex, the set Q is convex. From Claim ISTT] 
= inf§n^ F(X) < F{Q) and, consequently, int Lp{F{Q)) 7^ 0, where intA represents the 
interior of the set A. Moreover, it is immediate to verify that 

c cn, / G int Lf{f{q)) c Lf{f{q)) c n. 

Claim 5.4. The gradient vector fields grad Fi , grad F2 and gradFs are Lipschitz on Q. 

Proof. From the first equality in fl29|) and Proposition 12.51 is immediate that the gradient 
vector field gradFi is Lipschitz. Now, the second equality in ( l29l) implies that: 

\hessF2{X){V,V)\ =2e-^''^\tT[{2I - X-^)V^]\, XeM, VeTxM. 

As trX > and using the Cauchy-Schwartz inequality, we obtain from the above equality 
that: 

\hessF2iX){V,V)\ <2\\2I - X-^WfWV^Wf, XeM, VeTxM 
Hence, using the definition of the metric we conclude that: 

\hessF2{X){V,V)\<2{2^/^+\\X-'^\\F)\\V^F, XeM, VeTxM. 
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Thus, as ||X ^\\f < ti X ^, from the latter inequahty along with the definition of fi, we 
conclude that: 

\hessF2{X){V,V)\ <A^ + 2Kn, X e n, \\V\\f = l- (40) 

On the other hand, using the third equality in fl29l) and the Cauchy-Schwarz inequality, we 
obtain: 

|hessF3(X) {V,V) \ = {ti {X-^VX~'^V) \ < \\X-^VX-^\\f\\VX-^\\f, X e M, V e TxM. 

Since is submultiplicative and reusing that < trX~^ above inequality implies 

that: 

|hessF3(X) {V,V) \ < (trX-^)^ X G M, V e TxM. 

Thus, 

|hessF3(X) I < kV, X en, \\V\\f = 1. (41) 

From (140|) and (HTl) we conclude that hess F2 and hess F3 are also bounded operators when 
restricted to Q. Therefore, from Proposition I2.5[ gradFi, gradF2 and gradFs are Lipschitz 

on n. □ 

We denote by Lj the Lipschitz constant of grad/j, i = 1, 2, 3 on Q. 
From Claim [5?T] the assumption hi of the Theorem 14.11 is satisfied with U* = {/}. The 
next result will be useful to assure that the assumption h2 of the Theorem 14.11 is satisfied. 

Claim 5.5. There exists a > and X G such that, 

Lf{F{X)) C B^{I) C miLF{F{Q))nC, 
where B^{I) := {X G E'l^ : d(X, /) < a}, C is defined m (|30l) and Q m (138|). 

Proof. Since U* = {/} and / 7^ Q, we conclude that / G int Lf{F{Q)) fl C. So, it is easily 
seen that there exists a > such that: 

B^il) CmtLF{F{Q))nC, 

proving the last inclusion. In order to prove the first inclusion, assume by contradiction that 
for every X G B^^^I) we have Lf{F{X)) fl (S"_,_\i?o-(/)) 7^ 0- Therefore, there exist sequences 
{X''}, {F^} C such that 

X'^eBM), limX^ = /, e Lf{F{X''))\BM), fc = 0,l,.... (42) 

fc— ^-oo 
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Since {X^} G B,{I) C Lf{F{Q)) we have Lf{F{X^)) C Lf{F{Q)) for A; = 0, 1, . . .. Hence, 
jyfcj f~ Lp{F{Q)). As Lp{F{Q)) is a bounded set, we assume (taking a subsequence, if 
necessary) that {Y^} converges to some Y G Lp{F{Q)). Thus, continuity of F along with 
the two last equations in ( H2|) imply that: 

< F(J), Y^I, 

and, since f/* = {/}, we obtain a contradiction, which proves the claim. □ 

From the above claim along with the convexity of the ball B„{I), we conclude that the 
assumption h2 is satisfied with c = F{X). 

Claim 5.6. There exist 5 > such that 

||gradF,(X)|| >5, VXgL^(F(Q)), j = 1,2,3, (43) 
where Q is defined in f p8|) . 

Proof. Let X G M. Using f l28|) along with the definition of the Riemannian metric, we have: 
||gradFi(X)|| = v^, 

||gradF2(X)|| = ^/tr{I + 46-2*"^^ tr X + Ae-^*''^ tiX^), 
||gradF3(X)|| = VtiX-^. 

Now, if X G Lf{F{Q)), then from the second inequality in (|39|1 we conclude that: 

||gradF3(X)|| > v^/«. 

It is easily seen that || gradF2(X)|| > i/n. Thus, the result follows by taking 6 = ^Jnj k. □ 
For checking that F satisfies the assumption h3 we will use the following notation 

B^{I) := {X G ^!^+ : ||X - I\\f < r}, (44) 

for the Euclidean ball with center in / and radius r. Since / G int Lp{F{X)), where X is as 
defined in Claim [^75| take r > such that 

5f (/) C Lf{F{X)). 

Using Claim [5751 we have Lp{F{X)) C Lf{F{Q)), with Q as in ( l38l) . Then in order to prove 
that F satisfies the assumption h3 it is sufficient to prove: 
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Claim 5.7. There exists 6 > such that 

\\y{X)\\ >6, Xe Lp{F{Q))\Bf{I), y{X) G d°F{X). (45) 

Proof. Let (J) be the Euclidean ball as defined in (04]). Take X e Li.(F(Q))\5f (/) and 
consider the active indexes set 

/(X) := {z:F{X) = F,{X),z = 1,2,3}, 

of F at X. Hence Claim [5?T] implies that 

f {3}, X e U,- 

I{X) = I {1}, {3}, or {1,3}, XEU2; (46) 
[{2}, {3}, or {2,3}, X e U^. 

So, we conclude from Claim \5M that in order to prove the claim, it is suffices to consider the 
following cases: 

a) XeDi = {XeU2: I{X) = {1, 3}}; 

h) X eD2 = {X eUs: I{X) = {2,3}}. 

Let us suppose, initially, that a) holds, i.e., X G Di. Take y{X) G d°F{X). Lemma [7.31 
guarantees that it exists a G [0, 1] such that: 

y{X) = agradFi(X) + (1 - a) gradF3(X). 

Now, using ( 128|) . the definition of the Riemannian metric and the latter expression, we have: 

|b(X)f = tr {al - {1 - a)X-'Y , 

which after simple algebraic manipulations becomes 

n 

iii/wir = E("-(i-")Vw)'' (47) 

where Xj{X) denotes the j*'^ eigenvalue of the matrix X. From (147|) it is easy to see that 
UX)r >(«-(!- a)A-L(X))^ +(«-(!- a)X^UX)Y • 
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Minimizing the second degree polynomial of the right hand side in the above inequality with 
respect to a, we obtain: 



v2 



\\y{X)f> (^"^^-^^\ ^mix(^)) (4g^ 

" {i + KLix)Y + ii + x~Ux)f 

Since X G -Di C U2, combining Claim [5?T] with Claim [5^ we have Amm(-^) < 1- Further- 
more, in this case detX > 1 which, combining with Amin(-^) < 1, implies that 

Xma.x{X) > 1. 

On the other hand, as detX > 1 equation (!33l) yields trX > n. Since X ^ B^{I) and 
trX > n, we conclude from above inequality and after some algebraic manipulations that 

< tr(X - J)2 = tr X^ - 2 tr X + n < nALx(^) " 2n + nALx(^), 

which implies that 



Amax(X) > VI + ry2n. 

Combining first the inequality in f p9|) with the inequalities f HHj) and the latter inequality, we 
obtain: 



\y{X)\\>6,= , ^ l/v/l + r/2n ^ ^ ^ y(X) G 9°F(X). (49) 



^(1 + Kf + (1 + i/^/TTVJ^y 



Now, let us suppose that the item h) holds, i.e., X G Z^i. Take y{X) G 9°F(X). Due to 
Lemma [7.31 there exists a G [0, 1] such that 

y{X) = agradFi(X) + (1 - a) gradF3(X). 

Thus, (1251) . definition of the Riemannian metric and the last expression imply that 

\\y{X)f = tr (a(-X - 2e~2trx^2) _ _ ^^j^jS _ 

Some algebraic manipulations in the last equality and taking into account that Aj(X) > 0, 
for all j = 1, . . . , n yields: 

||l/(X)|p > a^n + 2a(l - a)trX"^ + (1 - a)trX-^ (50) 

Due to X G -D2 C f/3 we have detX^^ > 0. Hence, using fl55]) . we obtain: 

trX"-^ > n, trX~^ > n 
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Since a G [0, 1], substituting two above inequalities into f l50|) we obtain 

||y(X)f>n, XeD2, y{X)ed°F{X). (51) 

Therefore, taking 5 = min{5, 6i, n} > 0, where 6 is given in the last claim, the claim follows 
from m. (H9|). (I5T1) and Claim [531 □ 

Thus Lf{F{Q))\Lf{F{X)) C Lf{F{Q))\B^{I), the assumption h3 is satisfied with 
6 = min{(5, 6i, 62} > 0. 

Summarizing, all assumptions of Theorem 4.1 are satisfied by taking Q = {X G S'^^ : 
trX-i < KTi}, q = diag(l/4, ...,1/4), c = F{X), where X is as defined in Claim 15.51 
and 5 > is defined above. Letting X° G and A > such that X^ G Lf{f{q)) and 
maXjg/Lj < Afc < A, the proximal point method, may be applied for solving the above 
nonconvex problem (1311) . 

Remark 5.2. In agreement with the claims [377] and \5.2l the function 

F{X) = max{ln det X, - 4 In det X + e"^*'^ - e'^", tr - n}, 

in the above example, is nonconvex (in Euclidean sense) when restricted to any open convex 
neighborhood containing its minimizer X* = L Therefore, the classical local proximal point 
method cannot be applied to minimize this function. From the second relationship in (|29l) 
together Claim lST^ we conclude that the function F is nonconvex in the Riemannian sense as 
well. Therefore the Riemannian proximal point method cannot also be applied to minimize 
this function either. 

6 Final Remarks 

We have extended the range of application of the proximal point method to solve nonconvex 
optimization problems on Hadamard manifolds, namely, in the case of the objective function 
being given by the maximum of a certain class of continuously differentiable functions. We 
have certified, through examples, that the class of minimization problems for which the local 
proximal point method can be applied is different from the class of minimization problem for 
which the classical local proximal point as well as the Riemannian proximal point method are 
applied. An interesting subject now is to extend the proximal point method to minimization 
problems where objective functions are Lower-C^ type. 
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7 Appendix 

In this section we will use notation as in Example 15.21 
Lemma 7.1. If X e M and V e TxM, then 

d (expy t{D exp^)^^p-i yV, Y + tV 



lim 

t^O Y-fX t 

Proof. Preliminarily, note that 

d (^expy t{D expx)e^p-' y'^> Y + tV^ =d (expy t{D exp^)^^p-^i yV, expy (exp5;^(r + tV))^ . 

Since the exponential map exp : TM — )■ M is smooth, it is in particular locally Lipschitz. 
Let Ux C M be a neighborhood of X such that TUx ~ Ux x TxM and the map exp is 
Lipschitz in TUx with constant K. Take 5 = S{X) > such that, for all Y G Bs{X) and 
te (0,(5), 



d {expy t{D exp;,)^^p-^i yV, expy {expy' {Y + tV))^< K\\n{t, Y) \\ , 



(52) 



where l-L{t^Y) = tD expx)ey^p^^yV — expy iY + tV). On the other hand, by the definition 
of the metric (l25ll 

\\n{t, r)f = tr [{t{Dexp^)^^^-.yV - expy\Y + tV)) Y-')\ 

The definition of expy^ in fl26l) and above equality yields: 

\n{t,Yy^ 



< tr 



t{D exp^)^^p-^i y\/ - In (/ + ty- yy-i/^) Y^'^^ Y'^ 
which, after simple algebraic manipulation, implies: 

■ {D expx\^^-. yV - Y^/^ In (/ + tY-^/^VY-^'^) Y^'^) Y'^ 



nt,Y) 


2 

< tr 


t 





(53) 



lim In (/ + tF- V = X-i/ VX 



It is easily seen, that: 

lim Dexp^) -iy\/ = V, 
Hence, combining inequality ( 153|) with the two latter equalities, we conclude that: 

n{t,Y) 



-1/2 



lim 

t-^OY^X 



t 



0. 



The, last equality along with ( !52|) imply the desired result. 



□ 
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Lemma 7.2. Let il C be an open convex set. If F is a locally Lipschitz function on il, 
then 

F°{X, V) = V), X en, V e TxM. 

Proof. Take X e Q and V G TxM. Since F is locally Lipschitz on Q and TM is locally a 
product, there exists 6 = 6{X) > such that TBs{X) ^ Bs{X) x M" and 

F{g{t,Y))-F{Y + tV) 



for all y G Bs{X), t G (0, 5). Note that the above inequality is equivalent to 



F{g{t,Y)) - F{Y) F{Y + tV)-F{Y) 



t t 
On the other hand, Lemma 17.11 implies that 



d{g{t,Y),Y + tV) 
<Lx . 



hm djgmi^^o, 

i->-0 g-i-p t 

which combined with the above inequality and definitions of the generalized derivatives yields 
the lemma. □ 

Lemma 7.3. Let Q C be an open convex set and I = {1, ...,m}. Let Fi : M ^ 'R be a 

continuously differentiable function on Vt for every i e I and F : M — )■ M defined by 

F{X) :=max,g,F,(X). 
Then F is locally Lipschitz on Q and the following holds: 

d°F{X) = conv{gradFi(X) : i G /(X)}, X G fi, 
where I{X) := {i : Fi{X) = i = 1, m}. 

Proof. Take X E Q. From Proposition 13.41 it is sufficient to prove that 

d°F{X) C conv{gradFi(X) : z G I{X)}. 
Let W G d°F{X). Then, definition of the set d°F{X) and ([25]) provides 
F°{X, V) > {W, V) = tr {V^"{X)W) , F G 
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where "^{X) = — IndetX. Combining Lemma 17.21 with the definition of the Euchdean 
generahzed sub differential of F at X, d%F{X), we conclude that 

^"{X)W G d°EF{X), 

On the other hand, 

C conv{VFi(X) : i G /(X)}, 

where /(X) ■= {i e I : Fi{X) = F{X)}, see [5j Proposition 2.3.12. Then, there exist a 
constant > for i G I{X) with X]tg/(x) = 1 such that 

■^"{X)W= J2 «iVF,(X). 

So, since ^"(X) is invertible and gradFj(X) = \E'"(X)~^V-Fj(X), the above equality yields 

W= «igradF,(X). 
ieiix) 

Hence W G conv{gradFj(X) : i G /(X)} and the result is proved. □ 
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